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DETAILED ACTION 



Claim Objections 

1 . Claim 75 is objected to because of the following informalities: The term "create" 
should be changed to "draw" to follow claim 74's preamble. Furthermore, an apparatus 
cannot not "create" a virtual object per say, but "generate" a virtual object. Appropriate 
correction is required. 



Claim Rejections - 35 USC § 112 

The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of 
making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and use the same and shall 
set forth the best mode contemplated by the inventor of carrying out his invention. 

2. Claims 18, 19, 43, 44, 64, 65, 104-107, are rejected under 35 U.S.C. 112, first 
paragraph, as failing to comply with the enablement requirement. The claim(s) contains 
subject matter that was not described in the specification in such a way as to enable 
one skilled in the art to which it pertains, or with which it is most nearly connected, to 
make and/or use the invention. 

The limitation of embedding one or more sensors within a virtual component in 
claims 18, 43, 64, 104 is not supported by the specification. Paragraph [0069] teaches 
embedding sensors in the tool itself; however, the specification does not disclose 
embedding sensors within a virtual component. Furthermore, it is unclear how sensors 
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would be embedded in a virtual component considering sensors are real and the 
component is virtual. 

3. Claims 111, 121-126, are rejected under 35 U.S.C. 112, first paragraph, as failing 
to comply with the enablement requirement. The claim(s) contains subject matter that 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 

The limitation of the virtual deformation tool being used to deform the geometry of 
the virtual component in claim 111 is not supported by the specification. Paragraph 
[0015] teaches the virtual component is a software representation of a physical 
component. Furthermore, paragraphs [0119]-[0121] teaches the virtual deformation 
tool's function is to deform regions of three-dimensional virtual objects, not the virtual 
component. Thusly, it is unclear how and why the virtual deformation tool deforms 
itself. 

4. Claims 112, 127-133, are rejected under 35 U.S.C. 1 12, first paragraph, as failing 
to comply with the enablement requirement. The claim(s) contains subject matter that 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 
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The limitation of the virtual smoothing tool being used to smooth the geometry of 
the virtual component in claim 112 is not supported by the specification. Paragraph 
[0015] teaches the virtual component is a software representation of a physical 
component. Furthermore, paragraphs [0122]-[0124] teaches the virtual smoothing tool's 
function is to smooth regions of three-dimensional virtual objects, not the virtual 
component. Thusly, it is unclear how and why the virtual smoothing tool smoothes 
itself. 

5. Claims 113, 134-140, are rejected under 35 U.S.C. 112, first paragraph, as failing 
to comply with the enablement requirement. The claim(s) contains subject matter that 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 

The limitation of the virtual spray-painting tool being used to spray said virtual 
component with virtual paint in claim 113 is not supported by the specification. 
Paragraph [0015] teaches the virtual component is a software representation of a 
physical component. Furthermore, paragraphs [0125]-[0127] teaches the virtual 
painting tool's function is spray various colors of virtual paint onto regions of three- 
dimensional virtual objects, not the virtual component. Thusly, it is unclear how and 
why the spray-painting tool sprays virtual paint on itself. 



Application/Control Number: 1 0/791 ,972 Page 5 

Art Unit: 2672 

6. Claims 114, 141-147, are rejected under 35 U.S.C. 112, first paragraph, as failing 
to comply with the enablement requirement. The claim(s) contains subject matter that 
was not described in the specification in such a way as to enable one skilled in the art to 
which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 

The limitation of the virtual texture creation tool being used to spray a texture on the 
virtual component in claims 114 is not supported by the specification. Paragraph [0015] 
teaches the virtual component is a software representation of a physical component. 
Furthermore, paragraphs [0016] briefly mention the texture creation tool's function is to 
"spray" spray texture onto a three-dimensional virtual objects, not the virtual component, 
but does not provide any further details. Thusly, it is unclear how and why the virtual 
texture creation tool sprays a texture on itself. 

7. Claim 162 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply 
with the enablement requirement. The claim(s) contains subject matter that was not 
described in the specification in such a way as to enable one skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and/or use the invention. 

The limitation of more than 6 degrees of freedom in claim 162 is not supported by 
the specification. Even though paragraphs [0041], [0042], and [01 1 1] refer to more than 
6 degrees of freedom, it unclear from applicant's disclosure as to what the other 
degrees of freedom pertain to. 
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The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter that the applicant regards as his invention. 

8. Claim 4 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. Claim 4 recites the limitation "software" in line 1 . However, 
the software alone cannot perform the operation without a computer, which has not 
been claimed in the independent or depending claims. There is insufficient antecedent 
basis for this limitation in the claim. 

9. Claims 16, 41, 62 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Claims 16, 41, 62 recite the limitation "lack of virtual 
depiction". It is unclear how a virtual component can lack virtual depiction since the 
virtual component is depicted on the display. There is insufficient antecedent basis for 
this limitation in the claim. 

10. Claims 18, 19, 43, 44, 64, and 65 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. Claims 18, 43, 64 recite the 
limitation "embedding one or more sensors within a virtual component" in lines 1-2. 
However, it is unclear how sensors would be embedded in a virtual component 
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considering sensors are real and the component is virtual. There is insufficient 
antecedent basis for this limitation in the claim. 



11. Claims 23, 48, 69 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Claims 23, 48, 69 recite the limitation "buttons, 
joysticks, scroll wheels, or foot pedals embedded in said virtual component" in lines 1-2. 
However, it is unclear how buttons, joysticks, scroll wheels, or foot pedals would be 
embedded in a virtual component considering buttons, joysticks, scroll wheels, or foot 
pedals are real and the component is virtual. There is insufficient antecedent basis for 
this limitation in the claim. 

12. Claims 74-78, 81-97 are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. Claims 74, 81, 84, 87, 95 recite the limitation 
"coincidentally" in line 4. It is unclear what this term means in regards to how the two 
input devices are used in conjunction with each other. There is insufficient antecedent 
basis for this limitation in the claim. 

Claim 75 recites the limitation "said grabbing tool" and "said pointing tool" in line 3. 
No grabbing tool or pointing tool has been described in the independent claim to 
understand what "said grabbing tool" or "said pointing tool" pertains to. There is 
insufficient antecedent basis for this limitation in the claim. 
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Additionally, claims 92-94 recites the limitation "virtual object". However, as 
understood from the claims as well as the specification, the "virtual object" is the object 
displayed on the screen where the "virtual component" as related to the input device 
acts on. Thus, is the "virtual object" claimed in claims 92-94 really the "virtual 
component" of the physical input device? There is insufficient antecedent basis for this 
limitation in the claim. 

13. Claims 79, 80 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Claim 79 recites the limitation "said grabbing tool", 
"said gripping tool" and "said pointing tool" in line 4. There is insufficient antecedent 
basis for this limitation in the claim. 

14. Claims 102-109 are rejected under 35 U.S.C. 112, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. Claim 102 recites the limitation "iconic virtual 
component only" in line 3, "virtual component only" in line 6 and "a virtual component 
lacking virtual depiction" in line 7. It cannot be ascertained as to what the scope of the 
claim is. The term "only" provides a contradiction of the limitations. Is the iconic virtual 
component a separate entity along with the virtual component, and does the iconic 
virtual component and virtual component have additional components? Furthermore, it 
is unclear how a virtual component can lack virtual depiction since the virtual 



Application/Control Number: 1 0/791 ,972 Page 9 

Art Unit: 2672 

component is depicted on the display. There is insufficient antecedent basis for this 
limitation in the claim. 

Additionally, claim 104 recites the limitation "embedding one or more sensors within 
said virtual component" in lines 1-2. It is unclear how sensors would be embedded in a 
virtual component considering sensors are real and the component is virtual. There is 
insufficient antecedent basis for this limitation in the claim. 

15. Claims 111, 121-126, are rejected under 35 U.S.C. 112, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. Claim 111 recites the limitation "said tool is 
used to deform the geometry of said virtual component" in lines 1-3. However, it is 
unclear how the deformation tool deforms itself. There is insufficient antecedent basis 
for this limitation in the claim. 

16. Claims 112, 127-133, are rejected under 35 U.S.C. 1 12, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. Claim 112 recites the limitation "said tool is 
used to smooth a surface said virtual component" in lines 1-3. However, it is unclear 
how the smoothing tool smoothes a surface of itself. There is insufficient antecedent 
basis for this limitation in the claim. 
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17. Claims 113, 134-140, are rejected under 35 U.S.C. 1 12, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. Claim 113 recites the limitation "said tool is 
used to spray said virtual component with virtual paint" in lines 1-3. However, it is 
unclear how the spray-painting tool sprays virtual paint on itself. There is insufficient 
antecedent basis for this limitation in the claim. 

18. Claims 114, 141-147, are rejected under 35 U.S.C. 1 12, second paragraph, as 
being indefinite for failing to particularly point out and distinctly claim the subject matter 
which applicant regards as the invention. Claim 114 recites the limitation "said tool is 
used to spray a texture on said virtual component" in lines 1-3. However, it is unclear 
how the texture creation tool sprays a texture of itself. There is insufficient antecedent 
basis for this limitation in the claim. 

19. Claim 159 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. Claim 159 recites the limitation "said handle" in line 1. There 
is insufficient antecedent basis for this limitation in the claim. 

20. Claim 162 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. Claim 162 recites the limitation "more than 6 degrees of 
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freedom". It is unclear what the other degrees of freedom pertain to. There is 
insufficient antecedent basis for this limitation in the claim. 

No prior art rejections have been made on claims 16, 23, 41, 48, 62, 69, 74-80, 94, 102- 
109 due to the unclear scope and severity of the 35 U.S.C. 112, second paragraph 
rejections. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public 
use or on sale in this country, more than one year prior to the date of application for patent in the United 
States. 

Claims 1-4, 8, 28-31, 33, 34, 41, 53-55, 57, 58, 62, 81-87, 110, 115-117, 148-151, 

154-161 are rejected under 35 U.S.C. 102(b) as being anticipated by Pryor (US 
Publication No. US 2002/0036617 A1). 

Pryor teaches a method and apparatus for inputting position, attitude (orientation) 
or other object characteristic data to computers for the purpose of Computer Aided 
learning, Teaching, Gaming, Toys, Simulations, Aids to the disabled, Word Processing 
and other applications. 
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21. In regards to claims 1, 161 — 

Fig. 1a shows the basic invention of Pryor. A user (5), desires to point at an object 
(6) represented electronically on the screen (7) and cause the pointing action to register 
in the software contained in computer (8) with respect to that object (a virtual object), in 
order to cause a signal to be generated to the display (7) to cause the object to activate 
or allow it to be moved (e.g., with a subsequent finger motion or otherwise) (claim 1: 
generating one or more interface devices to alter and generate one or more two- 
dimensional or three-dimensional virtual objects) [01 09]. This system can provide 
highly accurate position and orientation information in up to 6 degrees of freedom (claim 
1 : wherein said devices can control N degrees of freedom of said virtual objects] 
claim 161 : wherein N degrees of freedom is 6 degrees of freedom) [0134]. 
Furthermore, the invention of Pryor can "alias" the interface devices, such as represent 
the input device, such as a pen, as a paintbrush on the screen (7) (claim 1: providing 
one or more three-dimensional virtual tools to a user for said spatial manipulation 
or said two-dimensional or said three-dimensional entertainment) [0275]. Fig. 9 
illustrates a means for aiding the movement of a person's hands while using the 
invention of Pryor in multiple degrees of freedom movement. Shown in Fig. 9a, joystick 
(905) has ball (910) attached to it's end in which the data from datums on the ball 
position is taken optically by a video camera in up to 6-axes [0382]. Two joysticks may 
be used (claim 1: generating one or more interface devices; associating said 
interface devices in conjunction with each other to alter one or more two- 
dimensional or three-dimensional virtual components) [0383]. A joystick is often 
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used for game control (claim 1 : providing a plurality of video game controllers to 
said user for spatial manipulation or said two-dimensional or said three- 
dimensional entertainment) [0382]. 

22. In regards to claims 2,3- 

Fig. 1a shows the basic invention of Pryor. A user (5), desires to point at an object 
(6) represented electronically on the screen (7) and cause the pointing action to register 
in the software contained in computer (8) with respect to that object (a virtual object), in 
order to cause a signal to be generated to the display (7) to cause the object to activate 
or allow it to be moved (e.g., with a subsequent finger motion or otherwise) (claim 2: 
said interface devices are digital input devices) [01 09]. The invention of Pryor 
permits the user to use a variety physical objects for the point action, where the sensors 
on the physical object are detecting by camera (10). The input device can even be the 
user s fingertip (12) (claim 3: said interface device are physical input devices) 
[0109]. 

23. In regards to claim 4 - 

The datums on an object can be known a priori relative to other points on the 
object, and to other datums, by providing it to a user via a CD ROM disc or other 
computer interface storage medium having this data [0143] (claim 4: virtual 
components are a software representation of physical input devices) 
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24. In regards to claims 8, 28, 53, 57, 148-150 - 

Fig. 6 illustrates other movements such as gripping or touching. Parts of the user 
such as the hands (said interface device), can describe motion or position signatures 
and sequence of considerable utility, e.g., grip, pinch (claims 8, 148: said interface 
devices is a grabbing tool), grasp (claim 53: said interface devices is a gripping 
tool), stretch, bend, twist, rotate (claim 149: wherein said grabbing controller is used 
to rotate said one or more virtual objects), screw, point (claim 28: said interface 
device is a pointing tool), hammer, throw [0331]. These actions can cause objects 
depicted on a screen to be acted on, by sensing the users hand relative to the human or 
the computer camera system or the screen [0333], Furthermore, as an example, if the 
hand (said interface device) is pulled away with in a certain time, it is interpreted to be 
a grasp, and the cow object (610) is moved to a new location on the screen where the 
user deposits it, for example, at position (650) (claim 57: wherein a first function of 
said gripping tool is to place said virtual objects in a 3D space, claim 150: wherein 
said grabbing controller is used to move said one or more virtual objects) [0334]. 

25. In regards to claims 29, 30 - 

As shown in Fig. 15, the input device controlled by the user (1549) may be a toy 
pistol (1550) (claim 29: said pointing tool has a physical form resembling a firearm, 
claim 30: said firearm is a gun) [0447]. 
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26. In regards to claim 31 - 

Referring to Fig. 13, a laser pointer (1320) can be used to obtain multiple points 
(claim 31: said pointing tool has a physical form resembling a laser pointer) [0430]. 

27. In regards to claim 33 - 

As shown in Fig. 6, the users hand can describe motion or position signatures and 
sequence of considerable utility, such a pointing (claim 33: said pointing tool has a 
physical form resembling a pointing hand) [0331]. 

28. In regards to claim 34 - 

Pryor teaches that the targeted tools could be simple targeted sticks (claim 34: said 
pointing tool has a physical form resembling a stick) [0271]. 

29. In regards to claims 54, 55 - 

Fig. 15 illustrates a sword play video game. As shown, player (1501) holds sword 
(1502) [0042]. This sword has a handle (claim 54: said gripping tool has a physical 
form resembling a handle, claim 55: said handle is a sword handle). 

30. In regards to claims 58, 157-159 - 

As shown in Fig. 3b, the user can pick up the tool (small paintbrush, large 
paintbrush, pen, pencil, etc.) (said gripping tool) and draw over the surface of a sheet 
of paper, surface of the display screen, or projection of computer display (claim 157: 
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said controllers is a drawing controller, wherein said controller is used to draw a 
stroke in said entertainment environment). The application software would not only 
trace the path of the tip of the targeted work tool, but also treat the tool as though it 
were a pen or paintbrush, etc., where the characteristics of the tool is found in the 
associated database [0274]. Furthermore, by drawing directly on a computer display or 
transparent cover over a computer display, the user can make the targeted tool (e.g. 
pen, paintbrush) manipulate the computer display and immediately get feedback on how 
the graphics are affected (claim 58: second function of said gripping tool is to draw 
one or more paths between two or more said virtual objects] claim 158: wherein 
said stroke is drawn freehand by said user in said entertainment environment) 
[0279]. In regards to claim 159, the tool, e.g., small, paintbrush, large paintbrush, would 
have a handle for the user to grasp in order to draw over the surface of a sheet of 
paper, surface of the display screen, or projection of computer display (claim 159: 
wherein said stroke is drawing using said handle by said user in said 
entertainment environment). 

31. In regards to claims 110, 115-117- 

Referring to Fig. 3, Pryor uses the example of an artist with tools, such as an eraser 
(claim 110: said 3D virtual tool is an eraser tool, wherein said tool is used to 
remove a region of a virtual surface) [0274], The artist would pick up the tool and 
normally use the tool over the surface of a sheet of paper, display screen or projection 
of computer display. The application software would not only trace the path of the tip of 
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the targeted work too, but also treat the tool as though it were the object, i.e. eraser. 
The exact characteristics would be found in the associated database. Extra 
parameters, such as the width of the line (claim 117) or color (claim 136) could be 
determined by keyboard input (plurality of controls, buttons) (claim 115: said tool 
has a plurality of controls to activate one or more functions, claims 116: wherein 
said plurality of controls comprises buttons, joysticks, scroll wheels, or foot 
pedals embedded in said tool, claim 1 17: wherein one of said functions is to 
change a size of a default erasing region) [0274]. 

32. In regards to claim 151, 154-156- 

Fig. 15 shows a sword (claim 151: wherein said controllers is a slicing 
controller) and pistol game (claim 154: said controllers is a pointing controller) 
where the players (1501) (1549) may use a toy pistol (1550) or sword (1502) that is 
viewed by stereo camera system (1510) [0447]. It is noted that both the sword and 
pistol is typically pointed at the screen [0453]. The player's hand (1575) or holster 
(1520) and pistol (1585) or sword (1502) may be targeted with one or more targets. On 
the screen in front of the player is a video display of the OK corral (and/or other imagery 
related to the game) with "bad guys" such as represented by computer graphics 
generated image (1535) [0447]. To play the game, the player draws his gun or sword 
when a bad guy draws his and shoots (claim 154: wherein said controller is used to 
shoot said one or more virtual objects, claim 155: wherein said pointing controller 
is used to select said one or more virtual objects in said entertainment 
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environment) or attacks, such as slicing [0448]. The computer game software controls 
the "bad guys" (1535) coming in to view or leaving the scene [0447]. Thus, when the 
player shoots or attacks the "bad guys" (1535), the computer game software removes 
the "bad guys" (1535) from view, i.e., rearranging or relocating the virtual objects (claim 
151: wherein said controller is used to slice and relocate said one or more virtual 
objects in said entertainment environment; claim 156: said pointing controller is 
used to grab or rearrange said one or more virtual objects in said entertainment 
environment). 

33. In regards to claim 160 - 

Fig. 4 illustrates a car driving game. Target (420) is placed on the steering while 
which can be seen by stereo pair of cameras (405) and (406). As the wheel is turned, 
the target moves in a rotary motion that can be transduced accordingly, or as a 
compound x and y motion by the camera processor system means in computer (41 1). 
The target (420) can alternatively be attached to any object that we chose to act as a 
steering wheel (421) such as the wheel of a child's play dashboard toy (425) (claim 160: 
wherein one of said controllers is a navigation controller, wherein said controller 
is used to navigate said user in said entertainment environment) [0294]. 

34. In regards to claims 81-87 - 

Fig. 6 illustrates parts of the user, such as the hands can describe motion or 
position signatures and sequences of considerable utility [0330]. Some natural actions 
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of this type are grip, pinch, grasp (claim 83: grabbing a first extremity), stretch (claim 
85: stretch said virtual object), bend, twist (claim 86: twist said virtual object), rotate 
(claim 82, 83: rotate said virtual object), screw, point, hammer, throw, and other 
natural actions learned in life (claim 81; change placement of a virtual object, claim 
84: deform a virtual object) [0331]. The method and system of Pryor detects the 
position of one finger, two fingers of one hand, one finger of each hand, two hands 
(claims 81, 84, 87: using two or more of said physical input devices 
coincidentally), or relative motion/position of any of the above with respect to the 
human or the computer camera system or the screen [0333]. These actions can cause 
objects depicted on the screen to be acted on [0334]. Thus, two hands can be used 
where one hand performs one action, such as grabbing (claims 83, 86: grabbing with 
said first grabbing tool), while the second hand performs a second action, such as 
rotating (claim 83: rotating with grabbing tool) or twisting (claim 86: twisting with 
grabbing tool). 

35. In regards to claims 90, 91 - 

Fig. 15 shows a sword and pistol game where the players (1501) (1549) may use a 
toy pistol (1550) (claim 91: pointing tool) or sword (1502) (claim 91: grabbing tool) 
that is also viewed by stereo camera system (1510) [0447]. It is noted that both the 
sword and pistol is typically pointed at the screen (claim 90: using two or more of said 
physical input devices coincidentally) [0453]. The player's hand (1575), holster 
(1520) and pistol (1585) or sword (1502) may be targeted with one or more targets. On 
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the screen in front of the player is a video display of the OK corral (and/or other imagery 
related to the game) with "bad guys" such as represented by computer graphics 
generated image (1535) [0447]. To play the game, the player draws his gun or sword 
when a bad guy draws his and shoots or attacks, such as slicing [0448] (claim 90: 
specifying a point claim 91 : using a first pointing tool and a second grabbing tool 
to specify a point). The computer game software controls the "bad guys" (1 535) 
coming in to view or leaving the scene [0447]. Thus, when the player shoots or attacks 
the "bad guys" (1535), the computer game software removes the "bad guys" (1535) 
from view. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

36. Claims 9-12, 32, 35-37, 56 are rejected under 35 U.S.C. 103(a) as being 

unpatentable over Pryor (US Publication No. US 2002/0036617 A1). 

Pryor teaches the limitations of claims 9-12, 35-37, 56 with the exception of 

disclosing the grabbing tools (claims 9-12), pointing tools (claim 33, 35-37), and gripping 

tools (claim 56) resembling specific physical forms. However, Pryor discloses using 

retro-reflective material [01 1 1] or targets [01 13] adhered to input devices. 
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With the aid of Fig. 6, Pryor teaches the act of grabbing where the thumb (601) 
and the first finger (602) are near an object such as a 3D graphic rendition of a cow 
(610) displayed on the screen (615). As the fingers are converged in a pinching motion 
depicted as dotted lines (620), the program of computer (630) recognizes this motion of 
fingernails (635) and (636) seen by cameras (640) and (641) connected to the computer 
which processes their image, as a pinch/grasp motion and can either cause the image 
of the cow to be compressed graphically [0334]. Although Pryor is silent about other 
grabbing tools, Pryor discloses using retro-reflective material [01 1 1] or targets [01 13] 
adhered to input devices. Therefore, such material can be adhered to kitchen tongs, 
pincers, scissors, and or tweezers that mimic the same pinching motion as the thumb 
(601) and first finger (602) (claims 9-12). 

Parts of the user, such as the hands can describe motion or position signatures 
and sequences of considerable utility, such as pointing [0331]. This action can cause 
objects depicted on a screen to be acted on, by sensing [0334]. Thus, similar to the 
grabbing tools, the retro-reflective material [01 1 1] or targets [01 13] can be adhered to 
input devices, such as a flash light, spray-paint can or camera to mimic the same 
pointing motion as the user's finger (claims 33, 35, 36). 

As shown in Fig. 15, the input device controlled by the user (1549) may be a toy 
pistol (1550) [0447]. Although Pryor is silent about using a glue-gun as a pointing 
device, it would have been obvious to interchange the toy-pistol with a glue-gun by 
again, using the retro-reflective material [01 1 1] or targets [01 13] to mimic the same 
point motion as the user's fingers (claim 37). 
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Additionally, parts of the user, such as the hands, can describe motion or position 
signatures and sequences of considerable utility, such as gripping or grasping [0331]. 
Fig. 15 illustrates a sword play video game. As shown, player (1501) holds sword 
(1502) [0042]. This sword has a handle. The retro-reflective material [01 1 1] or targets 
[01 13] can be adhered to input devices, such as a shovel handle to mimic the same 
gripping or grasping motion as the user's hand with the sword (claim 56) (1502). 

These tools can be considered targeted tools, where a targeted work tool can be 
a toy model of the real world tool or the tool itself, helping the user immediately visualize 
the properties of the tool in the computer program [0236]. Thus, it would have been 
obvious to use such physical items in order to provide a realistic simulation of the users 
motion in regards to the graphic rendition on the screen (615). 

37. Claims 88, 89 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Pryor (US Publication No. US 2002/0036617 A1). 

Pryor teaches the limitations of claims 88, 89 with the exception of modifying an 
axis of rotation. However, Fig. 6 illustrates parts of the user, such as the hands can 
describe motion or position signatures and sequences of considerable utility [0330]. 
Some natural actions of this type are grip, pinch, grasp, stretch, bend, twist, rotate 
(claims 88, 89: rotate said virtual object), screw, point, hammer, throw, and other 
natural actions learned in life (claim 89: using tools to move virtual component to a 
desired location) [0331]. The method and system of Pryor detects the position of one 
finger, two fingers of one hand, one finger of each hand, two hands (using two or more 
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of said physical input devices coincidentally), or relative motion/position of any of 
the above with respect to the human or the computer camera system or the screen 
[0333]. These actions can cause objects depicted on the screen to be acted on [0334]. 
Thus, two hands can be used where one hand performs one action, such as grabbing 
(claim 89: grabbing with said first grabbing tool), while the second hand performs a 
second action, such as rotating (claim 89: rotating with grabbing tool). Although 
Pryor is silent about an axis of rotation, and axis of rotation would be needed for the 
user to rotate the virtual object. Furthermore, the invention of Pryor permits the virtual 
object to be moved or altered in position. Thus, when the virtual object's position has 
been altered, either rotated a certain degree, etc., the axis of rotation as then been 
modified (claim 88: modify an axis of rotation). 

38. Claims 92, 93 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Pryor (US Publication No. US 2002/0036617 A1). 

Pryor teaches the limitations of claims 92, 93 with the exception of denoted a point 
by intersecting virtual objects and the pointing tools being laser beams. However, from 
the rationale of claim 90, it would have been obvious that a point would be created if the 
"bad guys" of the video game illustrated in Fig. 15 were to overlap (claim 92: 
interesting first and second virtual objects to denote said point). Additionally, in 
one embodiment of Pryor, Pryor teaches using a laser [0642]. In conjunction with the 
rationale of claim 90, Pryor teaches "aliasing" objects as other objects [0281]. Thus, it 
would have been obvious to one of ordinary skill in the art to alias the laser as a device 
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with properties for the entertainment environment (claim 93: said first and second 
point tool resembles a laser beam). 

39. Claims 95-97 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Pryor (US Publication No. US 2002/0036617 A1). 

Pryor teaches the limitations of claims 95-97 with the exception of disclosing an 
input device to cut a virtual object. However, Fig. 6 illustrates parts of the user, such as 
the hands can describe motion or position signatures and sequences of considerable 
utility [0330]. Some natural actions of this type are grip, pinch, grasp (claim 97: grab 
virtual object), stretch (claim 97: length virtual object), bend, twist, rotate, screw, 
point, hammer, throw, and other natural actions learned in life [0331]. The method and 
system of Pryor detects the position of one finger, two fingers of one hand, one finger of 
each hand, two hands (claim 95: using two or more of said physical input devices 
coincidentally), or relative motion/position of any of the above with respect to the 
human or the computer camera system or the screen [0333]. These actions can cause 
objects depicted on the screen to be acted on [0334]. Thus, two hands can be used 
where one hand performs one action, such as grabbing, while the second hand 
performs a second action, such as lengthening. 

Additionally, as shown in Fig. 15, a sword video game can incorporate the method 
of Pryor. It would have been obvious to one skilled in the art that the sword (1502) held 
by player (1501) can virtually cut the "bad guys" within the game [0441]-[0442]. 
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Furthermore, Pryor teaches that the invention allows the user to "alias" any object 
with any other object [0281]. Thus, the hands of the user can be aliased as a sword 
that can cut the virtual objects (claims 96, 97: cut virtual objects). 

40. Claims 152, 153 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Pryor (US Publication No. US 2002/0036617 A1). 

Pryor teaches the limitations of claims 152 and 153 with the exception of teaching 
using a handle with a laser beam as a slicing controller. However, in one embodiment 
of Pryor, Pryor teaches using a laser [0642]. In conjunction with the rationale of claim 
1 51 , Pryor teaches "aliasing" objects as other objects [0281]. Thus, it would have been 
obvious to one of ordinary skill in the art to alias the laser as a sword in the sword game 
described via Fig. 15 if a sword in not available (claim 152: said slicing controller's 
physical shape is a handle and one of its virtual component is a laser beam). 
Furthermore, in regards to claim 153, Fig. 6 of Pryor illustrates parts of the user, such 
as the hands can describe motion or position signatures and sequences of considerable 
utility [0330]. Some natural actions of this type are grip, pinch, grasp, stretch, bend, 
twist, rotate, screw, point, hammer, throw, and other natural actions learned in life 
[0331]. These actions can cause objects depicted on the screen to be acted on [0334]. 
Thus, as in the rationale of claim 152, the handle shaped slicing controller as claimed 
can be aliased as the hands described in Fig. 6, to perform the same functions of 
dropping objects (claim 153: said handle shaped slicing controller is used to drop 
objects). 
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41 . Claims 5-7, 13-15, 17, 20, 21 , 38-40, 42, 45, 46, 59-61, 63, 66, 67 are rejected 
under 35 U.S.C. 103(a) as being unpatentable over Pryor (US Publication No. US 
2002/0036617 A1) in view of Harvill et al. (US Patent No. 6,222,523 B1). 

Pryor teaches the limitation of claims 5-7, 13-15, 17, 20, 21, 38-40, 42, 45, 46, 59- 
61, 63, 66, 67 with the exception of teaching the virtual component coinciding with the 
tool's physical form. However, Harvill et al. attaching a tactile stimulus to the users 
body when a virtual operator, created by movements of the physical operator, 
encounters a virtual object defined by a computer. 

Referring to Fig. 6 of Pryor, Fig. 6 illustrates other movements such as gripping or 
touching. Parts of the user such as the hands (claims 15, 17, 40, 42, 61, 63: said tool), 
can describe motion or position signatures and sequence of considerable utility, e.g., 
grip, pinch (claims 15, 17: grabbing tool), grasp (claim 61, 63: gripping tool), stretch, 
bend, twist, rotate, screw, point (claim 40, 42: pointing too/), hammer, throw [0331]. 
These actions can cause objects depicted on a screen to be acted on, by sensing the 
users hand relative to the human or the computer camera system or the screen (claim 
20, 45, 66: said controlling position comprises integrating said virtual component 
within environment, claim 21 , 46, 67: said controlling environment is a camera) 
[0333]. 

Fig. 1 of Harvill et al. is an overall perspective view of a data processing system (4) 
wherein movements of a part of a physical body (10) of an operator (12) are converted 
into a virtual operator (16) (claims 13, 38, 59: virtual form of said tool is an iconic 
virtual component) for manipulating a virtual object (20) represented within data 
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processing system (4) [col. 2, lines 65 - 68; col. 3 lines 1 - 4]. As seen, the virtual 
operator (16) takes the form of the input device, i.e. the hand of the operator (10) 
(claims 14, 39, 60: virtual form of said tool is a first iconic component coupled with 
a second virtual component that coincides with said tool's physical form] claims 
1 5, 40, 61 : virtual form of said tool is a virtual component coinciding with said 
tool's physical form). As seen, the virtual form takes the shape of a 2D object (claim 
5: virtual component is a 2D GUI, claim 7: virtual component is a cursor). The 
movements of a hand (22) of physical operator (12) are converted into a virtual operator 
(16) through a glove (24). Thus, as the hand (22) moves, the virtual operator (16) 
mimics the movements on the display (claims 17, 42, 63: altering the relationship 
between said tool and its corresponding 3D virtual component mapping said tool 
to said corresponding virtual component). It would have been obvious to one of 
ordinary skill in the art to have the virtual form also take a 3D shape since the data 
processing system (4) converts the movements of a physical body (10) of an operator 
(12) and the movements are within a 3D space (claim 6: virtual component is a 3D 
GUI). As shown in Fig. 2 of Harvill et al., a programmable interval timer (28) receives 
data from a CPU (32) of data processing system (4) through a bus (36). Internal timer 
928) provides a signal to a stimulus circuit (40) through a bus (44) whenever virtual 
operator (16) encounters virtual object (20) (claims 17, 42, 63: controlling position of 
said corresponding virtual component to one of said virtual objects) [col. 3, lines 
15 - 22]. Although Harvill et al. is silent about generating an iconic form when the 
virtual component is close enough to react with the virtual objects, the virtual operator 
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(16) reacts with the virtual object (20), thus altering it's state when near the virtual object 
(20) (claims 17, 42, 63: generating an iconic form when corresponding virtual 
component is close enough to react to virtual objects). 

Therefore, it would have been obvious to one of ordinary skill in the art to include 
the conversion method of Harvill et al. to display a virtual operator coinciding with the 
physical form of the input device of Pryor so the user can visually see the relationship 
between the input device and the virtual object on the screen, providing an easier 
means of manipulations of the virtual object on the screen. 

42. Claims 22, 24-27, 47, 49-52, 68, 70-73, 98-101, 118-120 are rejected under 35 
U.S.C. 103(a) as being unpatentable over Pryor (US Publication No. US 2002/0036617 
A1) in view of Kumar et al (US Patent No. 6,222,465 B1). 

Pryor teaches the limitations of claims 22, 24-27, 47, 49-52, 68, 70-73, 98-101, 
118-120 with the exception of disclosing a virtual menu. However, Kumar et al. teaches 
a system and method for manipulating virtual objects in a virtual environment where the 
user can select options for a menu. 

Referring to Fig. 6 of Pryor, Fig. 6 illustrates parts of the user such as the hands, 
can describe motion or position signatures and sequence of considerable utility, e.g., 
grip, pinch, grasp, stretch, bend, twist, rotate, screw, point, hammer, throw [0331]. 
These actions can cause objects depicted on a screen to be acted on, by sensing the 
users hand relative to the human or the computer camera system or the screen [0333]. 
Furthermore, Pryor allows the user to "alias" any object with any other object (claim 98: 
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virtual components) [Pryor: [0281]. It would have been obvious that the first action 
mode as disclosed in claims 26, 51, 72, 120 is the function of the aliased object 
corresponding to the function of the object that it is aliased of, e.g. if a finger is aliased 
as a pencil, the finger's first action mode is to draw like a pencil (claims 26, 51, 72, 120: 
wherein said first action mode is a default action mode). 

Similarly, Kumar et al. teaches a system and method for manipulating virtual 
objects in a virtual environment, for drawing curves and ribbons in the virtual 
environment, and for selecting and executing commands for creating, deleting, moving, 
changing, and resizing virtual objects (claim 100: change said plurality of virtual 
components) in the virtual environment using intuitive hand gestures and motions [col. 
1, lines 65-68; col. 2, lines 1-4]. Referring to Fig. 6 of Kumar et al., when the active 
gesture (42) is presented (claims 22, 47, 68: plurality of controls to activate one or 
more functions; claims 99, 101 : virtual menu activated), a main control menu is 
displayed on the display unit (12) in the virtual environment (60). An exemplary main 
control menu (100) and a variety of exemplary sub-menus (200)-(400) for invoking 
different modes of operation of the system (8) are shown (claims 24, 49, 70, 118: 
virtual menu consisting of one or more choice for said user to choose from) [col. 
5, lines 17-23]. Furthermore, the exemplary sub-menus (200)-(400) would allow the 
user to alter the functions of the virtual components (claims 25, 50, 71, 119: toggle 
between a first action and a second action). The main menu (100) includes menu 
commands that corresponds to a particular mode of the system (8) (claim 98: using a 
virtual menu to map) [col. 5, lines 29 - 31]. 
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Therefore, it would have been obvious to one of ordinary skill in the art to include 
the virtual menu of Kumar et al. with the manipulation of virtual objects method and 
system of Pryor because the method and system of Pryor allows the user to "alias" any 
object with any other object [Pryor: [0281]]. Thus, virtual menus would be ideal so the 
user can choose from options to alias the object. 

43. In regards to claims 27, 52, 73 

The invention of Pryor uses weapons, such as swords and pistols as shown in Fig. 
15. With the combined invention of Pryor in view of Kumar et al., it would have been 
obvious that the virtual menu of Kumar et al. provide the option to choose from a 
plurality of virtual weapons since games involving weapons makes use of the invention 
of Pryor. Furthermore, objects can be aliased [Pryor [0281]], thus optional and 
additional weapons besides the virtual component of the physical input device can be 
used within the game of Pryor allowing for customization within the game [Pryor [0441]- 
[0455]]. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

Kobayashi et al. (US Patent No. 6,628,264 B1) 
Harvill et al. (US Patent No. 5,986,643) 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michelle K. Lay whose telephone number is (571) 272- 
7661 . The examiner can normally be reached on Monday - Friday, 7:00am - 4:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Michael Razavi can be reached on (571) 272-7664. The fax phone number 
for the organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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Patent Examiner * A 
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